Task-Agnostic Exploration via Policy Gradient of a Non-Parametric State Entropy Estimate

In a reward-free environment, what is suitable intrinsic objective for an agent to pursue so that it can learn optimal task-agnostic exploration policy? this paper, we argue the entropy of state distribution induced by finite-horizon trajectories sensible target. Especially, present novel and practical policy-search algorithm, Maximum Entropy POLicy optimization (MEPOL), policy maximizes non-parametric, $k$-nearest neighbors estimate entropy. contrast known methods, MEPOL completely model-free as requires neither any nor model transition dynamics. Then, empirically show allows learning maximum-entropy in high-dimensional, continuous-control domains, how facilitates meaningful reward-based tasks downstream.